Many image or video processing systems perform operations such as rescaling, de-interlacing, or frame-rate conversion, that require the creation of data not present in the source video. Despite their increasing sophistication, the available methods all have limits to their capabilities.
One approach is to use a selection of estimation methods, each of which typically has an area of specialism in which it can be expected to outperform other methods. The task then is to combine the outputs of several different estimation methods in order to create the optimum composite image.
Several approaches to constructing the composite image are currently used. Examples are a median of estimated values, a weighted combination of estimated values, or adaption between estimated values based on the video content. Each approach has problems.
A median of estimated values is simple, and is effective in rejecting outliers in favour of less extreme values. However, it is prone to errors. It is not unusual to find that the correct value is an outlier, particularly when that value is generated by the particular specialism of one estimation method.
When using a weighted combination of estimated values it is difficult to choose an appropriate weighting scheme that is sufficiently robust.
Adaption is often based on motion or edge characteristics and is therefore prone to errors in measurements and in the contributing estimators. It also requires significant tuning of algorithm parameters to achieve good performance. Furthermore, all techniques tend to be closely linked and tuned to the particular estimators being used.